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ABSTRACT 

The purpose of this study was the design and application of 
procedures for evaluating the academic success of students involved 
in the Saddleback Conmunity College District assessment and placement 
program. Further, it was the purpose of this study to design and apply 
procedures for evaluation of the placement instrument cut-off scores. 

Based on a review of the literature and data from a District 
pilot assessment and placement project, seven specific evaluation 
elements were defined and applied. The use of scatter diagrams was 
found to be a particularly efficient means of de terming cut-off scores. 
This method was used to identify both two tiered STOP/GO and three 
tiered STOP /CAUTION /GO type cut-off scores. 

It was recommended that student withdrawal petitions carry an 
indication of the reason for the student's withdrawal from a class and 
that sufficient computer support be readied for a District-wide 
assessment and placement program. Also it was recommended that the 
assessment instrument be administered to a large number of classes for 
validation purposes and that any requirement for a measure of correlation 
between placement test scores and final course grades be discouraged. 
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Chapter 1 

INTRODUCTION 

Nature of the Problem 

In anticipation of State legislation that will "require a 
mandatory assessment, counseling, placement and follow-up program in 
the California Community Colleges" (Commission, 1986:6) the Saddleback 
Community College District began a pilot assessment and placement 
project in December 1985, A group of new students was asked to 
participate in this project. Those who consented were given a battery 
of assessment tests and counseling based on their test scores and 
academic goals. A follow-up study of the academic success of these 
students was needed to complete the final stage of the pilot project. 

Purpose of the Study 

The purpose of this study was the design and application of the 
procedures for evaluating the academic success of the students involved 
in a Saddleback Community College District assessment and placement 
program. Further, it was the purpose cf this study to design and apply 
procedures for evaluating the placement instrument cut-off scores. 

Method of Investigation 

The typical evaluation elements for assessment and placement 
programs were identified from a review of the literature. The 
applicability of these elements was tested using the final course 

1 
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of the students noted above who agreed to participate in the pilot 
project. A comparison was made between the academic success of those 
students who actually took the placement tests and those who decided 
not to complete the testing phase. 

The literature suggested the use of scatter diagrams for 
analysis of the placement cut-off scores. To provide data on which to 
demonstrate this mehtod a set of mathematics and English classes took 
the corresponding part of the assessment instrument as a class exercise. 
These students' ocores and final course grades were plotted on scatter 
diagrams and the analysis technique applied. 
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Chapter 2 
BACKGROUND AND SIGNIFICANCE 
Foundation for the Study 

One of the results of student unrest In the late 1960's and 
early 1970's was a significant reduction in college testing, orientation, 
and counseling activities (Rounds and Anderson, 1984:3). Students 
demanded the right to make their own decisions and the right to fail 
(McCabe, 1985 and Cohen, 1984:6). It became clear, however, that 
student preparation for college work was often less than adequate. Ill- 
prepared and ill-informed students often made poor academic choices 
and performed poorly as a result of these choices. The focus of the 
1980' s thus shifted to quality and accountability with the trend toward 
student needs assessment and placement testing (Hector, 1984:4),. 
Rounds (1984b: 11) concludes that "the Age of Assessment is upon us." 

Rounds and Anderson (1984b) conducted an exhaustive study of 
assessment practices in the California community colleges. Their study 
describes the wide variety of test instruments used and dominance of 
locally constructed tests for mathematics. This report concludes with 
a call for organized research which will validate the effects of what 
has beer done in assessment and placement. Specifically, evaluation 
of the cut-off scores used for placement and follow-up and retention 
studies are needed. It is exactly to these areas that this practicum 
was addressed. 
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Relationship of the Study to the Seminar 

Glasser (1972:309) believes that the central problem in 
education is the need for the individualization of instruction. A 
critical step in meeting this need is the assessment of each individual's 
learning style and ability followed by the appropriate choice or 
assignment of educational paths (Glasser, 1972:311). The District's 
pilot project was an attempt to begin this assessment process. Through 
proper course placement students have a better opportunity for learning 
to occur. Losak (1984:19) states that cognitive learning is made more 
difficult by undue stress and anxiety. He adds that when these 
components of the educational enviomment are reduced, the learning 
process is enhanced. That this is a significant problem at the community 
college can be seen in the observation by Robbins (1986:A3) that 
thousands of students drop out of community college each year because 
they are enrolled in classes that are too difficult for them. Effective 
assessment and proper placement will save many of these students by 
removing the stress and anxiety associated with taking classes for which 
they are not prepared. Assessment and placement is, thus, an integral 
part of the application of learning theory and, as such, is directly 
related to the Learning Theory and Applications Seminar. 

Significance of the Study to the Saddleback 
Community College District 

The results of this study will be incorporateci into the final 
report to the District Board of Trustees concerning all aspects of the 
pilot project. Knowledge of the evaluation procedures will allow for 
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more effective planning of the data collection needs and costs for 

the follow-up portion of a large District-wide assessment and placement 

program. 

Review of the Literature 

Rounds and Anderson (1984b) completec* xn exhaustive study of 
the assessment activities in California community colleges. Responses 
to their inquiries were received from 99 of the 106 colleges in the 
California system. They found that only seven colleges pointed with 
pride to the research carried on at their institutions, although 
such research was often the basis for lauding the programs at other 
schools. This is true even though it was felt that empirical data was 
the only defensible basis for assessment of these programs. Forty-seven 
colleges made comments categorized as describing technical needs for 
their programs. Twenty-four of these needs were further identified as 
research needs and fully one-third of these were related to the search 
for more appropriate cut— off scores for advising purposes. Four others 
observed the need for general student follow-up and longitudinal research 
and two colleges requested increased information about retention. 

Rounds (1984a) studied in depth the asses'^.ment practices at the 
four California community colleges most often identified with effective 
assessment and placement programs in the State by the other California 
community colleges. At Sacramento City College the testing instrument 
was considered very successful because it did a good job of predicting 
whether a student could earn at least a C grade. Follow-up studies on 
the five percent who refused to accept the placement recommendations 
indicated most were unsuccessful. At the time of the study. Sierra 
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College was in the process of validating the cut-off scores for its 
locally developed mathematics test. Follow-up studies at Sacramento 
City College found that assessed students were better retained than 
those who were not assessed in each of the 47 different classes studied. 
Another study compared the final grades of students who tested into 
English lA with chose who came up the ladder of previou;i courses. 
Indications were that the tested group had better success. Both 
Fulierton College and Victor Valley College were engaged in studies of 
the established cut-off scores. 

Clearly che establishment of test cut-off scores is a major 
concern of assessment research. That this is likely to be addressed by 
each college individually can be seen from the State of California 
Mathematics Diagnostic Testing Project (1986:2) which refuses to 
recommend cut-off scores based on State-wide performance on their tests. 
Their newsletter states "We do not provide these scores because we 
believe that cut-off scores used for placement purposes should be 
determined separately at each campus," The need for this activity is 
further supported ty the finding of Rounds and Anderson (1984a) that 
locally developed assessment instruments we^re more used than any single 
published instrument in all areas except reading. 

In an e'^'aluation of skills assessment at Victor Valley Coilcga, 
California Holton (1985) reports an upward trend in retention over the 
four semesters of assessment activity. She also concludes that 
assessment testing l y be a factor in the decline in enrollment of new 
students. At the time of the report the College was compiling 
expectancy tools for success in certain courses based on assessment 
scores. Success was defined as a grade of C or better. 
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Cohen (1984) describes the evaluation of the assessment program 
at Sanat Barbara City College, California. The effect of student 
performance was evaluated by comparing the student final course grade 
distributions in English, essential skills, ESL and mathematics classes 
between Fall 1982 (before the assessment program began) and Fall 1983 
using the chi-square statistic. The fourth week to end of the semester 
attrition percent was compared between these same two semesters. 
Generally an improvement of grade performance occurred and there was 
a notable drop in attrition. The effect on attrition remained somewhat 
in aoubt because of the overall College drop in ntcrition perhaps due 
to the newly instituted ten dollar per course drop fee. 

In a review of the Assessment Center at Sacramento City College, 
California Haase and Caffrey (1985) found that the percent of students 
passing the developmental English course essay examination was beginning 
to increase. The students who receive A's and B's in transfer level 
courses possess 13+ grade proficiency levels in both reading and English. 

Anthony (1985), in a review of the assessment and placement 
pr gram at Santa Ana College > California, found that tha course attrition 
percent dropped in most of the mathematics and English courses in which 
students were placed between Fall 1983 and Fall 193^. An attempt was 
made to determine the correlation between placemen^ test scores and 
final course grades. No correlation was for six of the nine 

courses studied. The correlation coefficients for the other three 
courses, which were all mathematics classes, ranged from 0.21 to 0.44. 
The entire study was viewed as framing guidelines for expanded 
participation in the program and not as an evaluation of the process. 
In a further review of this program Anthony and Slark (1986) found 
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that in 7 out of 25 selected courr>es tested students had a higher rate 
of success than did the untested. 

Hector (If 84) used the correlation coefficient to determine the 
effectiveness of certain placement tests in predicting final course 
grades in selected college level courses. A significant positive 
correlation was found in eleven of the twelve cases studied. Values 
ranged from 0.21 to 0.54. Cut-off scores were determined by using 
scatter diagrams to minimize the two errc . false positives and misses. 
Students receiving a grade of C or better in the course were considered 
successful. To improve faculty and student acceptance of the cut-off 
scores a three tiered system of placement recommendations was developed. 
The levels were STOP, CAUTION and GO. Students with scores in the 
CAUTION region were advised to look at their high school performance and 
other factors before deciding on course selection. 

In response to challenges to the placement cut-off scores used 
at DeKalb Community College, Georgia by external sources, Johnson (1984) 
used the Pearson product moment correlation coefficient to show a 
significant correlation between SAT scores and the placement test 
results. Coefficient values ranged from 0.64 to 0.76. 

Influence of the Literature Review on This Study 

Rounds and \nderson (1984b) set -he stage for this study with 
their call for investigations that address cut-off score selection, 
student follow-up and retention. Following Anthony's (1985) example 
it was the purpose of this study to provide guidelines for an expanded 
assessment and placement process rather than to evaluate the process 
at this early stage of development. 
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Several studies (Anthony (1984), Hector (1984), Anthony and 
Slark(1986)) used the Pearson product moment coefficient of correlation 
to compare course grades with placement test scores and consequently 
this measure was used in this investigation. It was common in the 
literature for grades of C or higher to be considered an indication of 
successful student placement (Hector (1984), Holston (1985) and 
Rounds (1984a)), and this was taken as the comparable measure in this 
investigation. Hector's (1984) use of the scatter diagram to determine 
the cut-off scores was the technical basis of the similar effort in 
this investigation. Simple percents were used extensively by Cohen 
(1984), iiaase and Caffrey (1985) and Holton(1985) to compare the 
performance between different groups of students. That technique was 
also adopted here. 
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Chapter 3 
PROCEDURES 

Evaluation of the Academic Success of Assessment 
and Placement Participants 

Participants 

A member of the Admissions Office staff was asked by the 
Assessment and Placement Pilot Project Coordinator to invite new 
college students to participate in the project and take the assessment 
test battery. Contact was made with each new college student who applied 
for admission while this staff member was on duty between Ncv^ember 4 and 
November 27, 1985. Of the 131 students asked to participate. 124 agreed. 
The Admissions Office estimated that 650 new students were admitted 
during this portion of November. The set of students who agreed to 
participate in the pilot project was divided into two groups. The 
first group (called the program group) consisted of those students who 
took the assessment tests and followed the placement recommendation. 
The second group (called the control group) was composed on those who 
failed to attend any of the testing sessions and those tested students 
who placed themselves contrary to the counselor's recommendation. 

Data Collection 

A memorandum was sent to each ma^»iematlcs and English instructor 
o^^ students who agreed to participate in the pilot project that requested 
the students* final course grades. In those cases where there was no 
response, the appropriate Division Dean was contacted and supplied the 
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necessary information from the grade collection sheet submitted by the 
instructor. All assessment test scores were provided by the Pilot 
Project Coordinator. 

Analysis of the Data 

For both the program and control groups, the percent of those 
who enrolled in a mathematics class and completed it with a grade of 
C or better was calculated. The mathematics assessment and placement 
was considered effective if the program group's percent was higher than 
that of the control group. 

Because English placement already required completion of a 
prerequisite course or passage of an English Department assessment 
examination, the English assessment and placement was considered 
effective if the completion percent (calculated in the same manner as 
for mathematics) of the program group was at least as high as the 
control group's completion percent. 

Evaluation of the Placement Instrument 
Cut-off Scores 

Participants 

Five mathematics and six English classes taught by instructors 
interested in the pilot project gave the appropriate part of the 
placement instrument as a class exercise at the beginning of the Spring 
1986 semester. 

Data Collection 

The tests described above were scored and placement recommendations 
noted. These materials were not returned to the students. In addition 
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to the test scores, the final course grades for each of these students 
was needed. A memorandum was sent to each instructor whose class 
participated in this project that requested a copy of their final grt.de 
sheet. In the event an instructor did not respond, the appropriate 
Division Dean was contacted and supplied a copy of the grade collection 
sheet fubmitted by the instructor. In order to compare the pilot 
project placement instrument for English with the other current 
placement devices, the transcript for each student whose class took the 
English placement test was obtained from the Records Office. 

Analysis of the Data 

For each class that took a placement test, the relationship 
between the final course grade and student placement test score was 
calculated using the Pearson product moment coefficient of correlation 
(Mendenhall, 1983;433) that was defined to be 

r = SS /;SS SS 

xy X y . 

A value of r greater than or equal to 0.4 was interpreted to mean that 
the placement tests provided some significant prediction of future 
•jaccess in that particular course. 

As a means of evaluating the cut-off scores used for placement, 
two dimensional (test scores versus coarse grades) scatter diagrams 
(Byrkit, 1980:363) were constructed as suggested by Hector (1984:8) for 
each test group. Cut-off scores were found that yielded ten percent 
or less misses. A miss was defined to be a student who earned a D or 
F in a course in which the assessment instrument recommended placement. 
The percent of false positives was calculated for these cut-off scores. 
A false positive was a student who scored below the cut-off level for a 
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course but was successful in it. If the rate of false positives was 
ten percent or less, these cut-off scores would have placed 80 percent 
of the students correctly. A recommendation was then made to adopt 
these new cut-off scores. Otherwise, no change was recommended for the 
cut-off scores* 

Following the suggestion of Hector (1984) cut-off scores for 
each test group using a three tiered system of placement recommendation, 
STOP/CAUTION/GO, that yielded ten percent or less misses and ten percent 
or less false positives were found. This was not originally proposed in 
the practicum proposal but was considered a valuable addition to the 
investigation based on Hector's (1984) reported success. 

The three concurrent mechanisms for English placement (pilot 
project placement, English Department test and placement due to 
passing the prerequisite course) were compared using the results of 
the pilot project exams given in the mathematics and English classes 
noted above. After looking at the student's transcript and the list 
of students who participated in the pilot project, the most probable 
method of entry for each student was determined. The pilot project 
recommendations were compared with those of the other two methods. If 
the number of discrepancies was less than twenty percent, the methods 
were considered equivalent. If they were thought not to be equivalent, 
a recommendation to the English Department to review the consistancy of 
the three mechanisms was considered appropriate. 

Limitations and Assumptions of the Investigation 

The numerical results of this study were limited specifically to 
the assessment instruments used in the pilot project and the curriculum 
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design at the Saddleback Connnunity College District during the the 
1985 — 1986 academic year. 

It was assumed that students who took the assessment instruments 
inade an honest effort to do well. 

It was assumed that all data analysis was to be interpreted 
as merely illustrative examples of the evaluation process and not an 
evaluation itself. 

Definition of Terms 
The following particular terms were used extensively in this 

study. 

Program group was those students who took the assessment tests 
and followed the placement recommendation. 

Control group was those students who failed to atff^nd any of the 
testing sessions and those tested students who placed themselves contrary 
to the counselor's recommendation. 

A miss was a student who earned a D or F in a course that the 
counselor, based on placement test scores, would have recommended 
placement. 

A false positive was a student who scored below the cut-off level 
for a course but passed it with a grade of C or better. 

The Pearson product moment coefficient of correlation, r , was 
defined to be 

r = SS VSS SS 
xy X y 

The STOP /GO cut-off score was such that a student scoring below 
that value was given the recommendation to not enter the class (i.e. STOP). 
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Instead a course was recoiraneded for which that score was a GO. 

The STQP/CAUTIQN/GO cut-off score was such that a student 
scoring below the STOP/CAUTION score was given the recommendation to 
not enter the class (i.e. STOP). Instead a course was recommended for 
which that score was a GO. A student who scored at or above the 
CAUTION/GO cut-off score was given the recommendation to enter the class. 
Otherwise the student scored in the CAUTION zone and was advised to look 
at their hig>^ school performance and other factors before deciding on 
the course selection 
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Chapter 4 



RESULTS 



Specific evaluation elements for an assessment and placement 



program were identified through a review of the literature. An example 
of each element was developed using data from the District's pilot 
assessment and placement project. 



Evaluation Element 1 — Mathematics Placement 

For both the program and control groups, the percent of those 
who enrolled in a mathematics class and completed it with a grade of C 
or higher was calculated. The mathematics assessment and placement was 
considered effective if the program group's success percent vas higher 
than the control group's percent. 



was applied to the 124 students who originally agreed to participate in 
the pilot project. Nine of the 34 students in the mathematics program 
group enrolled in mathematics courses and three completed their course 
with a grade of C oi better. The program group's success measure was 
33 percent. Twelve students in the mathematics control group of 90 
students enrolled in mathematics courses and three students completed 
their course with a grade of C or better. The control group's success 
measure was 25 percent. The mathematics assessment and placement was 
considered effective because the program group had a higher percent with 
success. 



Evaluation of the Academic Success of the Assessment 
and Placement Project Participants 



As an example of tiie application of this evaluation element, it 
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Evaluation Element 2 — English Composition Placement 

For both the program and control groups, the percent of those 
who enrolled In an English composition class and completed it with a 
grade of C or better was calculated. The English assessment and 
placement was considered effective if the program group's percent was 
at least as high as that of the control group. 

As an example of the application of this evaluation element, 
it was applied to the 124 students who originally agreed to participate 
in the pilot project. Ten of the 35 students in the English program 
group enrolled in an English composition course. Three of these students 
completed their course with a grade of C or better. The program group's 
success measure was 30 percent. Six of the 89 students in the English 
control group enrolled in an English composition course. Three of these 
students completed their course with a grade of C or better. The control 
group's success measure was 50 percent. The English assessment and 
placement was not considered effective because the control group had a 
higher percent of success. 

Evaluation of the Placement Instrument 
Cut-off Scores 

The following evaluation elements were applied to the placement 
instrument scores received by students errolled in certain mathematics 
and English composition classes who took the placement instrument as a 
class exercise. These evaluation elements were necessary to determine 
the validity of the tests and cut-off scores used for course placement. 
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Evaluation Element 3 — Value of Placement 
Instrument Scores as Predictors of Final 
Course Grades 

The Pearson product moment coefficient of correlation 
(M adenhall, 1983:433) was used to determine the correlation between 
the students* final course grades and scores on the placement instrument. 
A value of the correlation coefficient, r, greater than or equal to 
0,4 was interpreted to mean that the placement test scores provided 
a significant level of prediction of future course grades in that 
particular class. 

As an example of the application of this evaluation element, it 
was applied to the eleven classes (seven different courses) tha** were 
given the placement instruments as class exercises. Two of the 
mathematics courses showed a significant level of correlation. Two 
of the English courses had a significant level of correlation with the 
objective portion of the English placement test. No other significant 
correlation was fourd. The complete results were placed in Table 1. 

Evaluation Element 4 — Evaluation of the 
STOP/GO Cut-off Scores 

Two dimensional (test scores versus course grades) scatter 
diagrams (Byrkit, 1980:363) were constructed for each course in which 
students were tested. The STOP /GO cut-off scores for each course that 
yielded ten percent or less misses were found. If this score also 
provided ten percent or fewer false positives a recommendation was 
made to change the cut-off score to this value. Otherwise no change 
was recommended for the cut-off score. 

As an example of the application of this evaluation element, it 
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Table 1 

Correlation Between Placement Scores 
and Final Courae Gradea 



InatnuDent 


Course 


Nximber of 
Students 


Correlstlon 
Coefflci.ent, r 


Indlcstlon of 
Slgnlf Icsnce 


Mathematics 

Level I 


Arithmetic, 
Math 350 


31 


0.50 


Significant 


Mathematics 

Level I 


Beginning Algebrs, 
Math 351 


23 


0.38 


Not Significant 


Mathematlca 

Level I 


" '^ermedlste Algebrs, 

h 122 


10 


0.15 


Not Significant 


Mathematlca 

Level II 


Pre-cslculus, 
Math 2 


10 


0.54 


Significant 


Engllah 

Objective 


Baalc Granmar, 

Eng 300 


34 


0.45 


Significant 


Engllah 
Writing Sample 


Baalc Graammr, 

Eng 300 


34 


0.22 


Not Significant 


Engllah 

Objective 


Fundaacntala of 

CompoaltloQ, Eng 200 


34 


0.43 


Significant 


Engllah 
Writing Sample 


Fundaacntala of 

Coopoaltlon, Eng 200 


34 


0.23 


Not Significant 


English 
Objective 


Principles of 
Compoaltlon, Eng lA 


34 


0.08 


Not Significant 


English 

Writing Sample 


Prlnclplea of 
Compoaltlon, Eng lA 


32 


0.20 


Not Significant 



was applied to the seven courses which were given the placement 
instrument as class exercises. The scatter diagrams were constructed 
and placed in the Appendix. For three of the courses it was possible 
to find a cut-off score which yielded both missess and false positives 
at the ten percent level or less. The cut-off scores were the same as 
currently used for one of the courses and, thus, two recoiranendations for 
change were made. These results were summarized in Table 2. 



Evaluation Element 5 — Evaluation of the 
STOP/CAUTIQN/GO Cut-off Scores 

Two dimensional (test scores versus course grades) scatter 

diagrams (Byrkit, 1980:363) were constructed for each course in which 

students were tested. A three tiered cut-off score system was defined. 
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Tabu 2 20 

tvaluatlon of STOP/CO 
Cut-off 8cor«t 



Courts 


InatniMnt 


Nuabar of 
Studanta 


Exlatlng 

Cut-off 


Cut-off for 
lOZ Hlaaaa 


^eaultlng talae 
Poeltlvaa la &10Z7 


RacoMandatlon 


ArithMtlc, 
Hath 350 


Hathematlca 
Uval I 


31 


3 


7 


No 
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The STOP/CAUTION cut-off score that yielde.-l ten percent or ^ewer false 
positives was found. The CAUTION/GO cut-off score that yielded ten 
percent or fewer misses was found. These two scores defined the three 
tiered, STOP /CAUTION /GO, system. 

As an example of the application of this evaluation element, it 
was applied to the seven courses (eleven classes) which were given the 
placement instrument as class exercises. The scatter diagrams were 
exactly thore found for e'»^aluation element 5. The cut-off scores for 
each course using a three tiered system were placed in Table 3. 

Evaluation Element 6 — Evaluation of Placement 
Consistency Between the English Department 
Test and the District Placement Test 

The English course placement recommendation made by the District 

placement instrument was compared with the placement recommended for 

those who entered the tested classes \x the English Department test. 

The number of discrepancies was counted. If the number of discrepancies 

was less than 20 percent, the methods were considered equivalent. If 
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Table 3 

Evaluation of STOP /CAUTION /GO 
Cut-off Scores 



Course 


Instrument 


Number of 
Students 


STOP /CAUTION 
Cut-off Score 


CAUTION/GO 
Cut-off bcore 


Arithmetic, 
Math 350 


Mathematics 
Level I 


31 


2 


7 


Beginning Algebra, 
Math 351 


Mathematics 
Level I 


23 


12 


16 


Intermediate Algebra, 
Math 122 


Mathematics 
Level I 


10 


20 


23 


Fre-calculus, 
Math 2 


Mathematics 
Level II 


10 


14 


20 


Basic Granmar, 
Eng 300 


English, Objective 
and Writing Sample 


34 


ESL 


200 


Fundamentals of 
Composition, Eng 200 


English, Objective 
and Writing Sample 


34 


300 


lA 


Principles of 
Composition, Eng lA 


English, Objective 
and Writing Sample 


32 


200 


200 



they were thought not to be equivalent, a recommendation sent to the 
English Department co review the consistency of these two assessment 
devices was appropriate. 

As an example of the application of this evaluation element, the 41 
English 200 and English lA students placed using the English Department 
exam who took the District English assessment instrument as a class 
exercise were considered • There were 41.5 percent discrepancies. A 
memorandum to the English Department notifying them of the apparent 
inconsistency between the tests would have been appropriate. It would 
have pointed out that the assessment instrument placed 31.7 percent 
lower and 9.8 percent higher than the Department test. 
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Evaluation Element 7 — Evaluation of the Placement 
Consistency Between Prerequisite Course Success 
and the District Placement Test . 

The Znglish coursa placement recommendation made by the placement 
instrument was compared with the placement recommended for those who 
entered the tested classes through success in a prerequisite course. 
The number of discrepancies was counted. If the number of discrepancies 
was less than 20 percent, the methods were considered equivalent. If 
they were thovght not to be equivalent, a recommendation sent to the 
English Department to review the consistency of these two assessment 
devices was appropriate. 

As an example of the application of this evaluation element, the 
34 English 200 and English lA students placed by prerequisite course 
success who took the District English assessment instrument as a class 
exercise was considered. There were 73.9 percent discrepancies. A 
memorandum to the English Department notifying them of che apparent 
inconsistency between the two placement devices would have been 
appropriate. It would have pointed out that the assessment instrument 
placed 61.7 percent lower and 11.8 percent higher than thp prerequisite 
course ladder. 
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Chapter 5 



DISCUSSION, IMPLICATIONS AND 
RECOMMENDATIONS 

Discussion 

A set of evaluation elements which could form the basis for the 
evaluation of a large scale District assessment and placement program 
was defined and demonstrated by example in the previous chapter. These 
examples pointed to a few potential weaknesses with some of the 
evaluation elements. If these weaknesses were removed the value of the 
evaluation elements would be significantly Increased 

The evaluation of the academic success of the assessment and 
placement participants treated all students who withdrew as unsuccessful. 
It would be desirable to remove consideration of students who withdrew 
for personal reasons unrelated to their academic ability to complete the 
course. This would be possible if a reason for withdrawal was recorded 
for each student drop petition. 

As Anthony (1985) had previously observed, there was little 

correlation between the placement test scores and final course grades. 

Although the correlation coefficient is a common evaluative technique, 

its weakness in this application is not surprising. The correlation 

coefficient provides a measure of the degrs** to which one quantity (such 

as final course grade) changes in a proportional manner with another 

quantity (such a? placement test score) , If both quantities increase 

together with the same ratio of change (i,e, have a linear relationship), 

the correlation coefficient, r, equals 1, If there is no such linear 
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relationship, r equals zero. If there is a tendency for the quantities 
to increase together in only a vague linear fashion, r will range 
somewhere between zero and one. 

It would be expected that the higher a student's score on the 
assessment test, the higher would be that student's grade for the course 
in which he was placed. It is thus common practice to call upon the 
correlation coefficient to give a quantitative measure of this 
relationship. There is a danger here, however. Because of the limited 
number of final course grade categories available (i.e. A,B,C,D,F) an 
initial linear relationship may become bent as students with ever higher 
placement teqt scores receive final course grades of A. An example of 
this is present in the District pilot project data. The Basic Grammar 
(English 300) students who took the objective portion of the English 
composition placement test had scores which ranged from 5 to 35 even 
though the cut-off score to move on to the next higher composition course 
was 19. A nice linear relationship (and r close to 1) between grades 
and scores up to 19 might be expected, but beyond that each student 
would be expected to receive an A. Even though the test scores increase, 
the final course grade can not increase and the previous linear 
relationship is broken and the value of r decreases. 

This is not the only potential problem with using correlation 
in the context described above. Because of the wide variety of grading 
practices, final course grades may be a poor choice as a measure of 
student achievement. Aubrecht (1979:3), for this reason, found grades a 
poor choice as a measure of student progress on which to judge teacher 
effectiveness. 

The purpose of the assessment test used in this investigation 
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was to measure readiness for a course. It had no ability to judge 
student motivation, dedication, desire and personal availability to do 
well in the course. It thus seems unreasonable to ask more from the 
testing instrument than to provide a threshold past which the student 
is declared competent to enter into the study of a particular level of 
material. 

With all of this in mind it is doubtful that the correlation 
coefficient between placement test scores and final course grades 
(evaluation element 3) should be retained as an evaluation element. 

The use of scatter diagrams to determine cut-off scores, as 
suggested by Hector (1984), was shown to be a straight forward process. 
Concern for the inflexibility of a single STOP /GO cut-off score was 
countered with the introduction of a three tiered STOP/CAUTION/GO system. 
Although the recommended cut-off scores are easy to calculate, the entire 
set of scores should be examined as a whole for consistency before 
changes are adopted. For example, using the data from the pilot project, 
the three tiered system yielded lower required readiness skills for 
Principles of Composition (English lA) than for its prerequisite course. 
Fundamentals of Composition (English 200). This situation might suggest 
reviev of the appropriateness of the assessment instrument for English lA 
placcsient . 

Implications 

This study has given the framework for the evaluation of the 
District assessment and placement program. It can thus be used as a 
planning tool for District management to determine the allocation of 
resources necessary to complete such a task. Although the evaluation 
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techniques are relatively simple, a large amouat of data would have to 
be collected and manipulated efficiently to allow success of the program 
on a District-wide scale. 

In each case were the evaluation of a specific assessment 
program was described in the literature, the program was found to have 
a positive influence on student achievement and retention. Such an 
impact is thus likely in this District. This will result in more 
efficient usp of the institution's educational resources and the students* 
time and effort. Ultimately an even further improvement in the 
educational process of the Saddleback Community College District should 
be realized. 

The literature clearly points to the need of all California 
community colleges to enter into the task of assessment and placement 
programs and placement cut-off score establishment. This study should 
provide assistance to other colleges new to this student service area. 

Recommendat J ons 

Several recommendations are offered as measures which can 
increase the effectiveness and convenience of the evaluation process. 

The assessment instruments should be administered to a large 
number of classes over the next several semesters. Only in this way can 
valid placement cut-off scores be established. 

Student withdrawal petitions should carry an indication of the 
reason for the action (i.e. ^^ersonal, academic, etc.). With this 
information in hand the academic success of the program participants can 
be more accurately determined. 

No attempt should be made to correlate placement test scores 
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with final course grades. As discussed earlier, the correlation 
coefficient is not appropriate for this task. Also it is not che job 
of the test to predict the level of eventual success but just the 
readiness for the content of a particular course* 

A computer program should be acquired that can search the student 
data base and pull out the final course grades for students indentified 
as participants in the assessment and placement project* The methods 
of data collection used in this small pilot project would strangle 
any large scale effort* 
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APPENDIX 

PLACEMENT TEST SCORES VS. FINAL COURSE GRADES 
SCATTER DIAGRAMS 
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Figure 1 

Scatter Diagram for Arithmetic (Math 350) Students 
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Figure 2 

Scatter Diagram for Beginning Algebra (Math 351) Students 
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Figure 3 

Scatter Diagram for Intermediate Algebra (Math 122) Students 
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Scatter Diagram for Pre-calculus (Math ^ St mts 
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Figure 6 

catter Diagram for Fundamentals of 
Composition (Eng 200) Students 
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Scatter DiagLam for Principles of 
Composition (Eng lA) Students 
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